Skip to content

Use in-place serialization of container elements#1296

Merged
feldergast merged 1 commit intosstsimulator:develfrom
leekillough:inplace_container_serialization
Apr 15, 2025
Merged

Use in-place serialization of container elements#1296
feldergast merged 1 commit intosstsimulator:develfrom
leekillough:inplace_container_serialization

Conversation

@leekillough
Copy link
Contributor

@leekillough leekillough commented Apr 15, 2025

This uses in-place serialization of container elements. For example, during deserialization, a map key is serialized, the map entry is created with a default value, and then the map entry is serialized. This allows pointer tracking to work.

(It still builds the containers up from beginning to end in one pass -- if necessary, a two-pass algorithm could be used, where all keys are (de)serialized first, map entries with these all of these keys are created, and then a second pass goes through all of the values and (de)serializes them in the order of the keys, so that the map entries' pointers, iterators and references are fixed when the map values are deserialized, and they can refer to each other and not get invalidated by insertion; this is more likely to be an issue with std::unordered maps than ordered maps, since std::map and std::multimap are stable with respect to iterators and references.)

Since different container categories (maps, multimaps, sets/multisets, std::forward_list, std:vector<bool>, and the remaining std::vector, std::deque and std::list) each require different calls to create the entries, the serialize_impl specializations for serialize_insertable.h list each supported container separately, by splitting the specialization container type with T<Ts...> so that T can be compared with standard container templates using the is_same_template_v<> utility template. This may be simpler to understand than the previous SFINAE technique, but it also allows if constexpr() specialization of the code for different container types, without having to repeat common code multiple times, and without having to create a new trait to test whether a type belongs to a certain standard container. if constexpr() is optimized away at compile-time, so there is no performance penalty for doing it this way.

@github-actions github-actions bot added AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) AT: CLANG-FORMAT FAIL labels Apr 15, 2025
@github-actions
Copy link

github-actions bot commented Apr 15, 2025

CLANG-FORMAT TEST - FAILED (on last commit):
Run > ./scripts/clang-format-test.sh using clang-format v12 to check formatting

@github-actions github-actions bot added AT: CMAKE-FORMAT PASS and removed AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) labels Apr 15, 2025
@github-actions
Copy link

github-actions bot commented Apr 15, 2025

CMAKE-FORMAT TEST - PASSED

@github-actions github-actions bot added AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) AT: CLANG-FORMAT PASS and removed AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) AT: CLANG-FORMAT FAIL labels Apr 15, 2025
@github-actions
Copy link

github-actions bot commented Apr 15, 2025

CLANG-FORMAT TEST - PASSED

@github-actions
Copy link

github-actions bot commented Apr 15, 2025

CMAKE-FORMAT TEST - PASSED

@leekillough leekillough force-pushed the inplace_container_serialization branch from e5577ca to af25ffe Compare April 15, 2025 08:47
@github-actions github-actions bot added AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) and removed AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) labels Apr 15, 2025
@github-actions
Copy link

github-actions bot commented Apr 15, 2025

CMAKE-FORMAT TEST - PASSED

@github-actions
Copy link

github-actions bot commented Apr 15, 2025

CLANG-FORMAT TEST - PASSED

@sst-autotester
Copy link
Contributor

sst-autotester commented Apr 15, 2025

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

@sst-autotester
Copy link
Contributor

sst-autotester commented Apr 15, 2025

Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED by label AT: PRE-TEST INSPECTED! Autotester is Removing Label; this inspection will remain valid until a new commit to source branch is performed.

@sst-autotester
Copy link
Contributor

sst-autotester commented Apr 15, 2025

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.6_sst-elements

  • Build Num: 1968
  • Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.6_sst-elements_MR-2

  • Build Num: 1924
  • Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.6_sst-elements_MT-2

  • Build Num: 1923
  • Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.6_sst-macro_withsstcore

  • Build Num: 870
  • Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.6_sst-core_Make-Dist

  • Build Num: 723
  • Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_Clang-Format_sst-core

  • Build Num: 677
  • Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-14-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-elements

  • Build Num: 490
  • Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-14-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-macro_withsstcore

  • Build Num: 295
  • Status: STARTED

Using Repos:

Repo: CORE (leekillough/sst-core)
  • Branch: inplace_container_serialization
  • SHA: af25ffe
  • Mode: TEST_REPO
Repo: SQE (sstsimulator/sst-sqe)
  • Branch: devel
  • SHA: ad7d76e1f96debf721ec577ebd005c3026a55edc
  • Mode: SUPPORT_REPO
Repo: ELEMENTS (sstsimulator/sst-elements)
  • Branch: devel
  • SHA: 036c12926d1b4f996cf909eb5b3be053735bab2a
  • Mode: SUPPORT_REPO
Repo: MACRO (sstsimulator/sst-macro)
  • Branch: devel
  • SHA: 42e85e1689d473c65fdbcc008ce57fd53fe80865
  • Mode: SUPPORT_REPO

Pull Request Author: leekillough

@sst-autotester
Copy link
Contributor

sst-autotester commented Apr 15, 2025

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.6_sst-elements

  • Build Num: 1968
  • Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.6_sst-elements_MR-2

  • Build Num: 1924
  • Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.6_sst-elements_MT-2

  • Build Num: 1923
  • Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.6_sst-macro_withsstcore

  • Build Num: 870
  • Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.6_sst-core_Make-Dist

  • Build Num: 723
  • Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_Clang-Format_sst-core

  • Build Num: 677
  • Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-14-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-elements

  • Build Num: 490
  • Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-14-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-macro_withsstcore

  • Build Num: 295
  • Status: PASSED

@sst-autotester
Copy link
Contributor

sst-autotester commented Apr 15, 2025

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
WARNING: NO REVIEWERS HAVE BEEN REQUESTED FOR THIS PULL REQUEST!

@sst-autotester
Copy link
Contributor

sst-autotester commented Apr 15, 2025

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@sst-autotester
Copy link
Contributor

sst-autotester commented Apr 15, 2025

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ feldergast ]!

@sst-autotester
Copy link
Contributor

sst-autotester commented Apr 15, 2025

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - This Repo does not support Automerge

@feldergast feldergast merged commit bcba296 into sstsimulator:devel Apr 15, 2025
7 checks passed
@berquist berquist added this to the SST V15.0.0 milestone Apr 23, 2025
@leekillough leekillough deleted the inplace_container_serialization branch May 11, 2025 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@feldergast feldergast feldergast approved these changes

Assignees

No one assigned

Labels

AT: CLANG-FORMAT PASS AT: CMAKE-FORMAT PASS

Projects

None yet

Milestone

SST V15.0.0

Development

Successfully merging this pull request may close these issues.

4 participants

@leekillough @sst-autotester @feldergast @berquist